Numpy Basics: Arrays and Vectorized Computation¶
Agenda:¶
You will learn about Numpy List, multidimensional array, arithmetic ndarray, indexing and slicing, element wise array, np.where
Video#1. Numpy Introduction¶
lingua franca(means medium of communication)
- Yaani jobhi Numpy mai hum data store karte hain wo
n dimensional array (ndarray)mai store hoga
- yaani agar hum python mai kaam karenge to humein loops ka sahaara lena parega, jabke yehi kaam hum Numpy mai karna chahen to hum ye single statement mai karsakte hain
contiguousmeans sharing a common border; touching
agar appne 100 by 100 ki array banayi hai to jab usko access karenge to yaani OS usko access karne jayega to wo ubko 1 saath le ayega qk wo 1 saath parhe hue hain, usko memry mai 1 jaga sai doosri jaga move nahi karna parhta,1 hee jaga sai usko data mil jata haijabke agar hum list ki baat karenge to list jo hai wo dynamic arrays hote hain usmai kuch portion of data kisi jaga parh wa hai to kuch portion of data kahin or parha wa hai to apne data doosri jaga wala access karna hai to apko doosri jaga jaana parega to ismai time lagega
Video#2. Numpy VS Python List¶
In [4]:
list1 = range(1000000)
list1
Out[4]:
In [2]:
import numpy as np
In [6]:
arr1 = np.arange(1000000)
arr1
Out[6]:
In [7]:
%timeit for i in range(1, 10): r = [x * 2 for x in list1]
In [8]:
%timeit for i in range(1, 10): r = arr1 * 2
Video#3. A Multidimensional Array Object¶
In [9]:
import numpy as np
Differnet Methods of Numpy:¶
1. np.zeros()¶
- is sai Numpy ki 1 array create hogi jismai saare elements zero honge, kis size ki array banani hai mujhe wo bhi batana parega
- e.g agar mai 4 by 4 ki array banana chahun to mai tuple dunga jiske through mai size dunga of the array
In [10]:
zerosArr = np.zeros((4, 4))
zerosArr
Out[10]:
2. np.ones()¶
- mai chahun to ones ki array bhi banasakta hun
In [11]:
onesArr = np.ones((5, 5))
onesArr
Out[11]:
3. np.empty()¶
- ismai desired shape kiarray create hojayegi or usmain data randomly kuch bhi hosakta hai
- ye kuch bhi randomly return karsakta hai, randomly kuch bhi data memory sai utha sakta hai, lekin desired shape ki array create hojayegi
In [16]:
emptyArr = np.empty((7,7))
emptyArr
Out[16]:
4. np.array(listName)¶
- is method ke zariye mai python list ko array mai convert karsakta hun
In [18]:
list1 = [1,2,3,4,5]
list1
Out[18]:
In [20]:
listToArr = np.array(list1)
listToArr
Out[20]:
5. np.arange()¶
- same as range in python
In [21]:
np.arange(1,100,10)
Out[21]:
Video#4. Arithmetic with ndarray¶
- agar hum scalar add karayenge to wo element wise add hojayega
Practical¶
Generating random number array using random.randn()¶
In [2]:
randArr1 = np.random.randn((10))
randArr1
Out[2]:
In [3]:
randArr2 = np.random.randn((10))
randArr2
Out[3]:
Creating 2 arrays to perform vectorization:¶
In [27]:
arr1 = np.arange(2, 10, 2)
arr1
Out[27]:
In [28]:
arr2 = np.arange(2, 10, 2)
arr2
Out[28]:
Adding two arrays:¶
In [29]:
add = arr1 + arr2
add
Out[29]:
Subtracting two arrays:¶
In [30]:
print(arr1)
print(arr2)
In [31]:
sub = arr1 - arr2
sub
Out[31]:
Multiplying two arrays:¶
In [32]:
mult = arr1 * arr2
mult
Out[32]:
Dividing two arrays:¶
In [33]:
div = arr1 / arr2
div
Out[33]:
Multiplying an array by scalar value:¶
In [34]:
print(arr1)
print(arr2)
In [35]:
multByScalar1 = 2 * arr1
multByScalar1
Out[35]:
inverse a matrix using scalar:¶
In [36]:
print(arr1)
print(arr2)
In [37]:
inverseMatrix1 = 1 / arr1
inverseMatrix1
Out[37]:
Check which elements in array are greater than 0 :¶
In [40]:
# since all elements are greater than 0, it is returning true
arr1 > 0
Out[40]:
Acces elements who are are greater than 0:¶
In [41]:
# agar numpy array ko as bool array pass kren to jo true hoti hain wo select hokar ajati hain
arr1[arr1 > 0]
Out[41]:
Video#5. Indexing & Slicing¶
Practical¶
In [42]:
arr1 = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
arr1
Out[42]:
In [43]:
arr1[4]
Out[43]:
Boolean Indexing:¶
In [45]:
# Accessing elements greater than 5
arr1[arr1 > 5]
Out[45]:
Fancy Indexing:¶
In [46]:
# agar mai isko 1 list pass kardun
arr1[[2,1]]
Out[46]:
Accesing multidimensional ndarray:¶
In [47]:
# creating random array of 10 by 10
randArr = np.random.randn(10, 10)
randArr
Out[47]:
Accessing 1st row from ndarry:¶
In [48]:
# 1st row uth kar ajayegi
randArr[0]
Out[48]:
Accessing 5th col from 1st row:¶
In [49]:
randArr[0][4]
Out[49]:
Accessing odd row:¶
In [50]:
# 1 sai lekar end tak rows chahiyen or 1 ko chor kar 1 chahiyen to step=2,
# or agar cols humein saare chahiyen to hum comma lagake, double colon lagake, starting, ending or step sixe skip karden
###########3 randArr[1:10:2,::] ##################
randArr[1:10:2]
Out[50]:
Accessing odd cols:¶
In [51]:
# agar mujhe tamaam rows chahiyen or shuru ke char cols chahiyen
randArr[::, 0:4]
Out[51]:
In [52]:
# Creating ones array of order 5 by 5
onesArr = np.ones((5, 5))
onesArr
Out[52]:
In [54]:
# 1st and last row and column ko skip karna hai
# to 1st and last row and 1st col and last col ko chor kar baqi ko hum zero (0) kardenge
onesArr[1:-1,1:-1] = 0
onesArr
Out[54]:
Video#6. Fast Element-wise Array functions¶
practical¶
In [66]:
arr1 = np.array([5, 10, 15, 20])
arr1
Out[66]:
1. np.sqrt()¶
- square root calculate karega kisi bhi number ki
In [59]:
sqrtArr = np.sqrt(arr1)
sqrtArr
Out[59]:
2. np.power()¶
- is function mein mai 1st arg mai array dunga 2nd arg mai bataunga ke kitni power raise karni hai
In [60]:
powerArr = np.power(arr1, 2)
powerArr
Out[60]:
3. np.maximum(1stArr, 2ndArr)¶
In [63]:
arr2 = np.array([50, 30, 50,-20])
arr2
Out[63]:
In [64]:
print(arr1)
print(arr2)
In [65]:
# ye dono array ke har member ko alag alag compare karega or jo max hoga usse lelega
maxArr = np.maximum(arr1, arr2)
maxArr
Out[65]:
Video#7. np.where()¶
#¶
- ternary operator ki tarah kaam np.where() main hota hai lekin differnce ye hai ke ye poore array pe apply hota hai yaani ke ye poori array ke har element ko check karega
- jab hum data science karte hain to data ata hai lekin baaz field missing hoti hain to un missing fields ke andar kuch default value daalna chahte hain to wahan pe ye
np.whereka function bohot kaam ata hai for exapmle apne 1 survey conduct kia or usmai apne employees ki salaries poochin lekin kuch employees ne apni salaries batayin 0 yaani wo apni salaries disclose nhi karna chahte, kuch ne values daaldi -1, baaqiyon ne values sahi daali
to ap ye karna chah rhe hain ke jahan pe salary 0 ya negatve arhi hai waha pe kuch default daalde e.g 25000 daalden
In [69]:
salariesArr = np.array([0, -1, 100000, 50000])
salariesArr
Out[69]:
In [71]:
# jahan pe salaries 0 hain wahan pe daalden 25000, otherwise salary ko hum as it is rehne den
analyzedSalariesArr = np.where(salariesArr <= 0, 25000, salariesArr)
analyzedSalariesArr
Out[71]:
In [72]:
# ya main yahan koi msg deskata hun us value ki jaga
msgSalariesArr = np.where(salariesArr <= 0, "Not OK", "OK")
msgSalariesArr
Out[72]:
Video#8. Mathematical & Statistical Methods¶
#¶
- ese methods joke boolean arrays pe kaam karte hain
###### mean() - e.g apke pass 1 bool array hai, mai ye dekhna chah rha hun ke usmai kitne elements hain joke true hain? to ap
sum()ka method use karen to apko pata chal jayega ke usmai kitne elements mojood hain qkTrue represents 1jab hum sum karenge to total count ajayega ######any() any()method check karega ke array mai koi element True to nhi hai? koi 1 element True to nhi hai? ######all()all()ka method check karega ke tamaam elements True to nhi hain? ##################################################################################################################
#¶
- iske elewah apke pass sorting ke methods bhi mojood hain
################################################################################################################
#¶
- numpy mai ese methods bhi mojood hain joke ye check karsakte hain ke unique elements konse hain array mai?
################################################################################################################
Practical¶
In [101]:
arr1 = np.array([10, 9, 7, 10, 7])
arr1
Out[101]:
(1) np.mean(arrName) OR arrName.mean()¶
In [79]:
testArr1 = np.mean(arr1)
testArr2 = arr1.mean()
print(testArr1)
print(testArr2)
(2) np.cumsum(arrName) OR arrName.cumum()¶
In [80]:
testArr1 = np.cumsum(arr1)
testArr2 = arr1.cumsum()
print(testArr1)
print(testArr2)
(3) np.cumprod(arrName) OR arrName.cumprod()¶
In [81]:
testArr1 = np.cumprod(arr1)
testArr2 = arr1.cumprod()
print(testArr1)
print(testArr2)
In [83]:
# mai ye check akrta hun ke arr1 mai jo bhi element 6 sai bara ho wo ajaye as a bool or True = 1
arr2 = arr1 > 6
arr2
Out[83]:
In [85]:
# doing sum of bool array
# it is returning 5 bcoz eac True is 1 so five True makes 5
boolArrSum = arr2.sum()
boolArrSum
Out[85]:
(4) np.any(arrName) OR arrName.any()¶
In [86]:
# mai ye check karta hun ke koi element True hai bhi ya nhi using any()
anyTrueArr = arr2.any()
anyTrueArr
Out[86]:
(5) np.all(arrName) OR arrName.all()¶
In [87]:
# ye check akrta hun ke tamaam elements True hain bhi ya nhi hain ?
allTrueArr = arr2.all()
allTrueArr
Out[87]:
In [90]:
print(arr1)
In [91]:
# lets make some modification to arr2
arr2 = arr1 > 7
arr2
Out[91]:
(6) np.sum(arrName) OR arrName.sum()¶
In [92]:
# ab hum agar sum karte hain to kia hoga ?
# now it is returning 3 bcoz there is three True so it sums to 3 each of whic equals 1
# 3 element aise hai joke True hain
boolArrSum1 = arr2.sum()
boolArrSum1
Out[92]:
In [93]:
# kia koi element aisa hai joke true hai? g bilkul hai
boolArrAny1 = arr2.any()
boolArrAny1
Out[93]:
In [94]:
# kia tamaam elements True hain ? G nhi
boolArrAll1 = arr2.all()
boolArrAll1
Out[94]:
Sorting arr1:¶
In [102]:
# first check that array is not sorted
print(arr1)
(7) np.sort(arrName) OR arrName.sort()¶
In [104]:
sortArr1 = arr1.sort()
print(sortArr1)
In [105]:
# arr1 sort hogyi
arr1.sort()
arr1
Out[105]:
(8) np.unique(arrName)¶
In [106]:
# finding unique elements using unique() ?
# ismai apko np.unique ka method use karna parega qk error arha hai
uniqueArr = arr1.unique()
uniqueArr
In [107]:
# now it works with np.unique()
uniqueArr1 = np.unique(arr1)
uniqueArr1
Out[107]:
Video#9. File Input Output¶
#¶
- Numpy main ap complete array ko bhi save karakte hain naa sirf single array ko balke multiple array ko file mai save karsakte hain or load bhi karsakte hain file sai
np.save()1st arg filename mangta hai or uski extension bhi mangta hain agar nhi denge extension to wo khud dedega or 2nd arg mai wo array ka naam mangta hai jisko ap save karna chahte hainlekin ye method sirf single array ko save kareganp.savez()agar ap multiple arrays save karana chahte hain to ye method use karenge1st arg mai filename phir comma separated arrays names that you wanna save- jab ap file load karenge to return mai apko 1 dictionary returun hogi jismai ap keys ke through un arrays ki values ko save karsakte hain ################################
Practical¶
In [108]:
#Create two arrays
arr1 = np.array([1, 4, 5, 7])
arr2 = np.array([8, 7, 9])
#print each of them
print(arr1)
print(arr2)
saving single array arr1:¶
In [109]:
# mai arr1 ko save karunga in a file named "testArr1" ab qk mai yahan extension nhi derha to ye automatically "npy" dedega
np.save("testArr1", arr1)
loading file named testArr1:¶
In [111]:
# laod karte hue mujhe file ki extension dena zaroori hai warna error ayega
loadArr1 = np.load("testArr1")
loadArr1
In [112]:
# now it works as we prvide extension
loadArr1 = np.load("testArr1.npy")
loadArr1
Out[112]:
Saving multiple arrays arr1 and arr2:¶
- ab mai yahan save() ka nhi balke
savez() ka method use karungaqk mai yahan multiple arrays save kara rha hun orextension bhi yahan npy nhi balke npz hogi
In [124]:
np.savez("testArr1AndArr2", arr1=arr1, arr2=arr2)
In [125]:
# now it is returning dict
# is sai hum values retrive karsakte hain
loadArr1AndArr2 = np.load("testArr1AndArr2.npz")
loadArr1AndArr2
Out[125]:
Retrieving value from dict:¶
In [127]:
# Jab uppar humne ye save ki thin to arr1 or arr2 diya tha key name nhi diya tha like arr1=arr1 and arr2=arr2 to ye cell..
# .. error derha tha now it works
loadArr1AndArr2["arr1"]
Out[127]:
Video#10. Linear Algebra functions¶
#¶
arrName1.dot(arrName2)product of two matrix calculate karegaarrName.inv()inverse of marixarrName.det()determinant nikaalsakte hain ############################################################
In [3]:
arr1 = np.array([
[24, 45],
[12, 56],
[44, 78]
])
arr1
Out[3]:
In [4]:
# arr1 is 3 by 2 of order
np.shape(arr1)
Out[4]:
In [5]:
arr2 = np.array([
[34, 56, 90],
[22, 12, 34],
])
arr2
Out[5]:
In [6]:
# arr2 is of 2 by 3 order
np.shape(arr2)
Out[6]:
(2) arrName1.dot(arrName2):¶
In [7]:
dotArr = arr1.dot(arr2)
dotArr
Out[7]:
In [9]:
print(arr1)
print(arr2)
(3) arrName.transpose()¶
In [11]:
# (3 by 2) ka matrix (2 by 3) mai convert hokar agya
transposeArr = arr1.transpose()
transposeArr
Out[11]:
In [15]:
traceArr = arr1.trace()
traceArr
Out[15]:
Video#11. Pseudo Random Number Generator¶
Random number ki zarurat q pesh ati hai? jab hum dataset load karte hain to datasets mai biasness ko khatam karne keliye aksar humne apne data ko randomize karna parhta hai, data ko shuffle karna parhta hai, taake kisi data ki specific sequence ki wajah sai biasness ko khatam karsaken
truly random number generate karna poosible nhi hai computer keliye qk computer kisi formula ke taht random number generate karta hai isiliye hum kehte hain ke ye pseodo-random number generations modules hain
- computer jab random number generate karta hai to wo kisi distribution ke taht karta hai, normal distribution bhi hosakti hai, gamma distribution bhi hosakti hai, uniform distribution bhi hosakti hai
- to ye tamaam distribution ke through random numbers generate karne ki facility mojood hai lekin 1 important baat ye hai ke jab hum random numbers use kar rhe hote hain to humne random numbers generator ko initialize karna parta hai, issiliye hum usko starting mai seed value dete hain, seed value ke through wo random numbers ko initilize karta hai
Practical¶
In [5]:
# seed value ap kuch bhi desakte hain
np.random.seed(7)
In [6]:
# ab mai normal distribution ke through 3 by 3 ka 1 matrix generate karwata hun randomly
# to ye eik random number generate hua normal distribution ke thorugh
np.random.normal(size=(3,3))
Out[6]:
In [7]:
# unifom distribution ke through bhi random number generate karsakte hain
np.random.uniform(size=(3,4))
Out[7]:
12. Numpy Advanced Array Manipulations¶
13. Reshape¶
- row major order or column major order ye bunyaadi farq hai ke array kis tareeqe sai internally stored hai
- reshape karte hue agar main order kehta hun C, to C mai, row major order main data store hota hai, e.g: agar mere pass 0 sai leakr 11 tak elements hain, agar ye C order main hain ya row major order main hain to ye banega 0,1,2, 1st row lekin agar ye F order main he to ye 0,1,2 column banega
Practical¶
In [19]:
arr1 = np.floor(np.abs(np.random.randn(4,5)))
arr1
Out[19]:
In [20]:
# by default isne row major order mai save kia hai, ismai pehli dimension 2 hai second dimension bhi 2 hai or last yaani 3rd..
# .. dimension 5 hai
arr1.reshape((2,2,5))
Out[20]:
In [24]:
# reshape karte hue mai column major order bhi batasakta hun
orderC = arr1.reshape((2,2,5), order='C')
orderC
Out[24]:
In [25]:
orderF = arr1.reshape((2,2,5), order='F')
orderF
Out[25]:
Video#14. Concatenate¶
- concatenate as an argument a set of arrays leta hai or unko concatenate karta hai either row wise ya col wise, iskeliye wo eik argument leta hai axis ka jo argument ye batata hai ke apne row wise concatenate karna hai ya col wise karni hai
- diagram ke through samjhte hain apke pass LHS par 2 arrays hain 1st array 2 by 3 ki hai jismain tamaam elements 1 hain or 2nd array 2 by 3 ki hai jismai tamaam elements 9 hain, agar hum in dono arrays ko concatenate karen, or arg hum pass karen axis ka 0, to iska matlab hai ke row wise concatenation karni hain, ismai hum dekhsakte hain ke cols ki tadaad 3 hee rhi rows ki tadaad 4 hogyi hai, jo cols they woh wohi rhe or jo rows thin wo stack down hote hue 2 sai rows 4 hogyi
- is doosri example main hum arg pass karte hain axis=1 or wohi same do arrays hain to ye col wise concatenate hogya
- concatenation mai ap axis ka arg pass karte hain 0 ya 1 to iske kuch conveninent functiins bhi hain
vstackorhstack vstackmatlab ap concatenation perform kar rhe hain oraxisapne0diya hai, yaani row wise concatenate kar rhe hainhstackmai ap concatenation kar rhe hain along cols ya axis=1 kardia apne is function ke through
practical¶
In [29]:
# create list 1 & 2
list1 = [[1,3,5], [7,9,0]]
list2 = [[11,13,15], [17,19,10]]
In [30]:
# convert list 1 & 2 to np array
arr1 = np.array(list1)
arr2 = np.array(list2)
In [34]:
# concatenate two arrays
# as a tuple both arrays will be passed
# concatenated row wise, and col wise respectively
row_wise = np.concatenate((arr1,arr2), axis=0)
col_wise = np.concatenate((arr1,arr2), axis=1)
print(row_wise)
print("=======================")
print(col_wise)
In [35]:
# concatenate two arrays using functions
# dont need to specify axis
row_wise1 = np.vstack((arr1,arr2)) # vstack >>> row wise
col_wise1 = np.hstack((arr1,arr2)) # hstack >>> col wise
print(row_wise1)
print("===================")
print(col_wise1)
Video#15 Split¶
- jese concatenate 2 arrays ko jorta hai split 1 array ko mukhtalif hisson mai torta hai
- isko hum is diagram ke through samjhte hain
- mere pass 1d array hai jismai 1 sai lekar 9 tak elements hain, mai ne isko split karte hue specify kia ke position 2 pe or position 6 pe, x array ko split karna hai, to ap dekh sakte hain ke hamare pass 3 arrays return hue
- pehla portion 0 sai start hote hue 2 pe end horha hai, doosra 2 sai start hote hue 6 pe end horha hai or teesra 6 sai start hote hue end tak jarha hai, to ye method specified positions pe apki array ko split kar rha hai
Practical¶
In [36]:
# create list
list1 = [1,4,5,6,7,9,1,4]
In [37]:
# convert list1 to array
arr1 = np.array(list1)
arr1
Out[37]:
In [39]:
# np.split() mai 1st arg mai us array ka naam dena hai jisko maine split karna hai, 2nd arg mai bataunga kin positions pe split
# ..karna hai, to ye 3 parts mai array break hogyi
# [1, 4, 5, 6, 7, 9, 1, 4] >>>>>> array elements
# 0, 1, 2, 3, 4, 5, 6, 7 >>>>>> indexes od elements
# | |
# mai ne kaha ke 2 or 5 par split kardo to ye un specific index par jakar split kardega
np.split(arr1, [2,5])
Out[39]:
2d Array¶
In [42]:
# agar mere pass 2d array hoti
list1 = [[1, 4, 5, 6, 7, 9, 1, 4], [11, 14, 15, 16, 17, 19, 11, 14]]
In [43]:
# convert list to 2d array
arr2d = np.array(list1)
arr2d
Out[43]:
In [45]:
# splitting arr2d col wise(axis=1)
# [
# [ 1, 4, 5, 6, 7, 9, 1, 4],
# [11, 14, 15, 16, 17, 19, 11, 14]
# ] 0, 1, 2, 3, 4, 5, 6, 7
# | |
np.split(arr2d, [4,6], axis=1)
Out[45]:
In [47]:
# splitting arr2d row wise(axis=0)
# [
# [ 1, 4, 5, 6, 7, 9, 1, 4],
# [11, 14, 15, 16, 17, 19, 11, 14]
# ] 0, 1, 2, 3, 4, 5, 6, 7
# |
# rows 2 parts mai break hojayegi if split along 0 axis
np.split(arr2d, [1], axis=0)
Out[47]:
Video#16 Tile & Repeat¶
- repeat ka function as an arg eik array leta hai, or us array ke har elemnt ko repeat karta hai specified number of times, e.g: mere pass eik array hai jismai elements hain 6,7,8 or mai isko repeat karta hun 2 times, main arg deta hun ke mujhe isse 2 dafa repeat karna hai, to array ka har element 2 times repeat hoga
- jabke tile ka function poore array ko tile karega ya repeat karega
- e.g:
- mere pass 2 by 2 ki 1 array hai agar mai isko col wise tile kart hunto mujhe ye is tarah dikhega
Practical¶
In [48]:
# create an array
arr1 = np.array([1,5,7,8])
arr1
Out[48]:
In [50]:
# I call np.repeat() call karta hun or 1st arg mai main array ka naam pass karta hun jisko mai ne repeat karna hai , or 2nd ..
# .. arg main mai ye batata hun ke kitni baar repeat karna hai
# to har element 3 times repeat hoga
repeatArr1 = np.repeat(arr1,3)
repeatArr1
Out[50]:
In [51]:
# to np.tile() poori array ko repeat karta hai
tileArr1 = np.tile(arr1,2)
tileArr1
Out[51]:
brodcasting: e.g agr hum 2 arrays par koi operation perform kar rhe hain or 1 array small zize ka hai, or 1 array large size ka hai to small size ka array apne ap ko khud hee expand karlega, is concept ko hum broadcating kehte hian
In [ ]: